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Abstract 



The stochastic knapsack has been used as a model in wide ranging applications from 
dynamic resource allocation to admission control in telecommunication. In recent years, a 
variation of the model has become a basic tool in studying problems that arise in revenue 
qq" , management and dynamic/flexible pricing; and it is in this context that our study is un- 

dertaken. Based on a dynamic programming formulation and associated properties of the 
C^- , value function, we study in this paper a class of control that we call switch-over policies — 

start from accepting only orders of the highest price, and switch to including lower prices 
as time goes by, with the switch-over times optimally decided via convex programming. We 
establish the asymptotic optimality of the switch-over policy, and develop pricing models 
based on this policy to optimize the price reductions over the decision horizon. 



1 Introduction 

The stochastic knapsack refers in general to a dynamic resource allocation problem in which 
a fixed amount of resource is allocated sequentially to random demands of multiple classes. 
The problem appears to have many different roots. In Derman et al (1972), a Markov deci- 
sion problem is formulated to determine how to assign fixed amount of different resources to 
sequential arrivals with random unit returns. In Prastacos (1983), a problem for sequential 
investment is analyzed which allows only one acceptance decision to be made over the horizon. 



*Research undertaken while an academic visitor at IBM T.J. Watson Research Center; also supported in part 
by NSF grant DMI-0085124 and Hong Kong RGC Grant CUHK4173/03E. 
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In the late 1980's, the stochastic knapsack was used as a model to study admission control in 
telecommunication networks, so-called loss networks in particular; refer to Ross and Tsang [16j, 
and Ross and Yao |17j . 

In recent years, one version of the stochastic knapsack has become a basic model in studying 
problems in the general area of revenue management. Here the capacity of the knapsack corre- 
sponds to a given amount of resource that can be used to fulfill customer demand, over a given 
time frame that is typically quite short. Some examples include: rooms in a hotel targeted for 
weekend tourists, seats on an airplane that must be sold before departure, fashion items at a 
department store designed for a particular season. Refer to Kleywegt and Papastavrou [TOtfTT]. 
Papastavrou el al. [15] . and Van Slyke and Young [20] . 

Our study here also falls into this category, although it is motivated by a new application. A 
major producer of personal computers (PC's) from time to time has to liquidate a substantial 
inventory of desktop or laptop computers returned from corporate leases. This involves a 
number of sales channels — including catalog direct sales, dealers/brokers, and on-line auction 
- which differ in both price and batch size. What is needed by practitioners involved in 
this process is a tool to support pricing decisions. Specifically, not only what type of "bids" 
(customer demand or offer to buy the PC's) to accept and when (in terms of time and available 
inventory), but also how to price the sales over time. Refer to more details in [H [5j. 

As evident from previous studies, with prices given, the problem of when to accept or reject 
which demand can be formulated and solved using dynamic programming (DP). This, however, 
does not in general lead to optimal policies that have simple and easy-to-implement structures; 
neither does it result in a tractable value function, upon which pricing optimization can be 
carried out. Our approach makes use of certain properties of the DP value function, such as 
concavity and submodularity (which have been familiar properties in other application contexts 
such as queueing control, e.g., Lippman [13]), which lead to a lower- and upper-orthant structure 
of the optimal policy (see Proposition [I]). This structure, in turn, motivates us to focus on a 
class of "switch-over" policies. 

The switch-over policy is executed in the following manner: it starts from accepting only 
orders of the highest price, and gradually switches to including lower prices as time goes by. 
There are several advantages in focusing on this class of policies: a) they are consistent with 
the lower- and upper-orthant structure associated with the optimal policy; b) they are easy 
to identify: the optimal switch-over times are readily derived through convex programming; 
and most importantly, c) they are practical for implementation: indeed they follow closely the 
prevailing practice in certain industry sectors. Furthermore, we can prove that the switch-over 
policy, while sub-optimal in general, is asymptotically optimal in the sense that the relative 



2 



error between the switch-over and the optimal policies goes to zero as the available inventory 
increases to infinity (along with the planning horizon). 

One aspect of our work that is a departure from previous studies is that we are concerned 
with setting the optimal prices (in addition to deciding order acceptance/rejection). Based 
on the switch-over policy, we formulate optimization problems, so that the reduced ("sales") 
prices over the decision horizon can be optimally determined, taking into account that the rate 
of demand is a (decreasing) function of the price. A closely related set of papers in revenue 
management, although not always making an explicit connection to the stochastic knapsack, 
studies problems that are similar to ours in both physical and mathematical aspects. These 
include Bitran and Mondschein pQ , Brummelle and Walczak [2] , Feng and Gallego [H [7] , Feng 
and Xiao [8], Lee and Hersh |12j . and Zhao and Zheng |21j . 

Briefly, the rest of the paper is organized as follows. In we start with a dynamic 
programming problem formulation, and bring out the structure of the optimal policy. In the 
next two sections, we focus on the switch-over policy, starting with the case of constant batch 
sizes in The general case of random batch sizes is studied in §01 and the asymptotic 
optimality of the switch-over policy is established in We then develop the pricing models in 
£JH and conclude with possible extensions in $71 

2 The Dynamic Programming Formulation 

Here is a formal description of our model. There are W units of inventory available to supply 
the orders at times n = 1, • ■ ■ , T, where T is a given integer, representing the planning horizon. 
The order (demand) that arrives in each period n takes the form of a bivariate random vector: 
(P n ,Qn), where the two components represent the unit offer price and the required quantity. 
Suppose (P n ,Qn) are i.i.d. across n, following a joint distribution: 

P[P n =p i ,Q n =j]:=e ij , i = l,...,m; j = l,...,W. (1) 

Here, we assume, for all i and j, pi > and ^ i Ylj ®ij ^ 1> with 

i j 

representing the probability that there is no order arrival in a period. 

In each period, our decision is, after observing the realized (P n ,Qn), whether or not to 
supply the order. If we do, a revenue of P n Qn is collected; otherwise, we earn nothing. Here, 
we assume that each order is either supplied in full, or not at all; i.e., no partial supply is 
allowed. In particular, if the inventory available upon an order arrival is less than the order 
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size, then no supply takes place. The objective is to maximize the expected revenue collected 
over the planning horizon of T periods and the total available inventory of W units. 

Let V(n, d) denote the expected revenue we can collect, under optimal actions, starting from 
period n (< T), with d (< W) units of inventory left. Then, we have the following dynamic 
programming (DP) recursion: 

V(n,d) = V(n + l,d)[9 + e(d)] 

+ J2J2 di r ma x{pd + v ( n + 1 > d -j)> v ( n + 1 > d )}> ( 2 ) 

i j<d 

where 

<~)i,/;: ( 3 ) 

i j>d 

Clearly, the first term on the right hand side of ([2]) corresponds to the case of either no arrival or 
the order size exceeds the available inventory; whereas each term under the double summation 
compares the two actions: accept (i.e., supply) the order, or reject it. If we supply the order, 
then we earn the revenue pjj, and proceed to the next period with j units less in the available 
inventory. In the last period, we have 

i j<d 

since clearly the best action is to supply any possible order using all the remaining inventory. 

The above DP is quite easy to solve - the overall computational effort is, after all, only 
0(TW). Short of any structural properties, however, the solution does not readily translate 
into a policy that is easy to implement. We need to pre-compute and store the V(n, d) values 
for all n = 1, ...,T and all d = 1, ...,W. Then, after observing the realized demand (P n ,Qn), 
we will supply it, if Q n < d and 

P n Qn + V(n + l,d-Q n ) > V(n + l,d); (5) 

and reject it otherwise. Furthermore, the lack of a tractable form of the value function with 
respect to the prices is a severe handicap when it comes to solving the optimal pricing problem. 

In the case that all orders are of unit size, the above problem can be reduced to a special 
case of a certain queueing control model in Lippman [13], and the value function, V(n, d), can 
be shown to be concave in d and submodular in (n, d). These properties have the following 
implications: 

• For each price type i, if the order is rejected in some state (n*, d*), then it is rejected in 
all "lower" states (n, d) < (n*,d*); if the order is accepted in some state (n*,d*), then it 
is accepted in all "upper" states (n, d) > (n*,d*). 
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To understand the above, we know from the DP recursion, if a type i order is rejected in 
state (n*, d*), then 

Pi < V{n*,d*) -V{n*,d* - 1). 
For any state (n, d) < (n*,d*), we have 

V(n*,d*) - V(n*,d* - 1) < V(n*, d) - V(n*,d - 1) < V(n, d) - V(n, d - 1), 

due to the concavity (the first inequality) and submodularity (the second inequality) of V . 
Hence, 

Pi < V(n,d) - V(n,d- 1), 

i.e., the same order should also be rejected in state (n, d) as well, which is the lower-orthant 
property. On the other hand, if the order is accepted in state (n*,d*), then it must be accepted 
in any state (n, d) > (n* , d*); for if it is rejected in state (n, d), then following the lower-orthant 
property, it must also be rejected in state (n*,d*). 

The above properties, stated in a slightly different but equivalent form below, are the basis 
for the switch-over policies that we shall focus on in the rest of this paper. 

Proposition 1 Suppose all orders are of the same, constant size. (Hence, without loss of 
generality, assume this constant size is unity, i.e., Q n = 1.) Then, the optimal policy has the 
following structure: for each inventory level d, there exist time epochs = to(d) < t\(d) < 
• • • < t m (d) = T such that a price type k order is accepted (rejected) if and only if n > tk_i(d) 
(n < tk-i(d)); and for each k, t/~{d) is decreasing in d. 

Throughout the paper, we use "increasing" and "decreasing" in the non-strict sense. 

3 The Switch-Over Policy 

The structural result in Proposition Q] suggests a threshold type policy as follows. For each type 
of order i, there is a critical state (rij, di) — supply the order if and only when time has reached 
rii or beyond and inventory is at least di. This policy, however, is still difficult to analyze (in 
terms of deriving an explicit objective function to be used for pricing optimization). What we 
do below is to reduce this two-dimensional threshold policy to a single dimension, in time only. 
Specifically, we start with accepting only orders with the highest price, until a time t%, when 
we start to accept orders of the top two prices, until a time t2, when we start to accept orders 
the top three prices, and so forth. We call this a "switch-over" policy. 
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It is important to note that the switch-over policy essentially follows the time epochs tt(d) 
(1 = 1,... , m) in Proposition [T] but ignores their dependence on the inventory level d. Conse- 
quently, it is suboptimal even in the context of Proposition [TJ 

In this section, we focus on the case of a constant (unit) batch size, treating the general 
case of random batch sizes in the next section. 

3.1 Optimizing the Switch-Over Times 

We let the switch-over times be the decision variables of an optimization problem, with the ob- 
jective to maximize the expected profit over the horizon - the same objective as in the dynamic 
programming formulation. (Note, however, since these time epochs are chosen independently 
of the level of the available inventory, the switch-over policy is in general sub-optimal.) To 
determine the best switch-over times, 

= to < h < t 2 < ■ ■ ■ < t m -i <t m = T, (6) 

(only t\, ...,t m _i are decision variables), we consider a continuous-time version of the original 
problem, with the order streams following independent Poisson processes with rates Aj, i = 
1, ...,m, and associated with the price pi, such that 

Pi > P2 > ■ ■ ■ > Pm- (7) 

Let denote the total number of order arrivals of types {1, 2, • • • , k} over the time interval 
(tfc_i,ifc]; N\k follows a Poisson distribution with mean (Ai + • • • + \k)(tk ~ tk—l)- Let 

XiPi H h XkPk / x 

Plk '= A 1 + - + A fc ' (8) 
denote the average unit price of the orders accepted (by the switch-over policy) over the time 
interval (ifc— ljtfc]. Then, we can write the objective function associated with the switch-over 
policy as follows: 

max p u Ep¥AiVi] + p 12 E[(W - Nu) + A N u ] 

m—l 



+ ■■■+ Plm E[(W - iV lfc) + A N lr 



k=l 



pn[W - E(W - Nn) + ] + pi 2 [E(W - Nn) + - E(W - N 1 - N ■ 
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m—l 



+ ■ ■ ■ + Pi m [E(W - Ni k ) + - E(W - Nik) + ] 
k=i fc=i 

m—l 

PiiW - (pn - Pl2 )E(W - Nn) + (p hm _i -pi m )E(W - N ik) + 

k=l 

m 

-pi m E(W-Y N ik) + - (9) 



k=l 
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Denote 



[i k :— \\t k + A 2 (tfc — *i) H + ^^(tfc — tfc-i) 

= Aiti + (Ai + A 2 )(t 2 -ti) + --- + (Ai + --- + A fc )(t fc -t fc _i), (10) 

which is the mean (as well as the variance) of N%\ + N\2 + • • • + iVi&. Let iV(/x) denote a Poisson 
random variable with mean \x. We can then turn the optimization problem in ([9]) into the 
following equivalent form: 



min (pn - pi 2 )E[W - + (pi 2 - Viz)^W ~ N(fi 2 )} + + 

0<ti<---<t m _i<T 

1 + 



+(pi, TO _i - Pi, m )E[W - iV(/i m _i)] + + pi m E[VF - A^(^ m )] + . (11) 
Denote the distribution function of N(fj,) as: 



i^ n ( M ):=P[iV(/x)<n] = ^| r e^; (12) 

k=0 

and define a function: 

H(ji) := E[W - JV( M )]+ = J^(VF - = W^wOO " M^V-i(m)- (13) 

fc=0 

Clearly, is decreasing and convex in fi, since (x) + is increasing and convex, and N(fi) is 

stochastically increasing and linear in [i (refer to [18J). 

From (|13p . we can derive explicitly H'(fii), which will become useful below. First, taking 
the derivative of F n (fj,) in (fl~2j) . we have, 

- fM := FM = F n _i(/i) - F„( M ) = -^-e-". (14) 

n! 

Hence 

F'C/i) = -Wf w (fi) - Fw^ifi) + vfw-i(p) = -F W -i{»), (15) 
taking into account that fw(fJ-) = (/■*)• As a by-product, the above also leads to 

H"(jn) = fw-l(f-Ll) > 0, 

confirming the convexity of H(fi). 
Denote: 

TTfc -=Pik ~Pl,k+i, yk-=tk-t k -i, k=l,...,m; (16) 
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with pi, m +i := 0, and recall to := and t m := T. Then, the optimization problem in (fTTj) can 
be expressed as follows: 

m m 

min ^ ■KjHbj.j) s.t. V"y^ = T; ye > 0, £ = 1, ...,m; (17) 

where the new decision variables are (ye)] and, following ()10p and (|16p . 

^ = Aiyi + A 2 y2 H h A^, (18) 

with 

A* := Ai + • • • + \t. (19) 

3.2 Solution Approach 

A shortfall of the above formulation is that the objective function in (|17|) is not separable in 
(yi) (although the constraint is). If we make (fj,i) the decision variables instead, then both the 
objective and the constraint will become separable. From (|18p . we can solve for (ye)'- 

y e = t , 1=1, ...,m; (20) 

A^ 

with fjLQ := 0. We can then turn the optimization problem in (|17p into one with (m) as decision 
variables as follows: 



mm 



s.t. 



][>tf( w ) (21) 
m 1 1 



< \i x < • • • < Vm] (23) 

where A~ , j := 0. Also note that in (|22p we have changed the original equality constraint (= T) 
to an inequality constraint (< T), taking into account that H(-) is a decreasing function, and 
1T£ > as evident from ([7l I8ll6p . Furthermore, as pointed out earlier, H(-) is convex; hence, 
we now have a separable convex programming problem. In addition, it has another appealing 
property as revealed in the lemma below. 

Lemma 2 The ratio of the coefficients in (j22j) to those in (|2Tj) . (-£- — a^")/ 71 "^' * s decreasing 
in £. 

Proof. We need to show: 

1-1 1 x 1,1 1 X 

" >-(l"-l )> e = 2,...,m. 
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The above inequalities simplify to: 



> , « = 2, m — 1; and > 1. (24) 



TT^-l A^ + i A£ + i 

To verify these inequalities, write 

/0£ := ^ AfcPfc and ^ = 

k=l 



First consider t^m. Straightforward algebra yields: 

pi-i\i - piXiki-x 

We want the above to dominate A^+i/A^. This is equivalent to 

Pi - Pi+i^i > Pi-i - PeM-i, 

or, 

pAe > Pe+iM - Pt^e-i = pAi - (pe - pe+i)Ai, 
which obviously holds, since pi > pe+i- Next, we show the last inequality in (|24[) . Since 

_ Pm _ Pm—1 Pm 

7Tm — T— , 7T m _l — - — , 

Iy m Ji m— 1 is -m 

we have 



7T m PmA 



m—1 



Pm—X^m Pm^- 



m—1 



We want the above to dominate A ^ 1 ; and this simplifies to 

PmAm ^ Pm— lA m p m A. m -.\j 

or p m A m > p m _iA m , i.e., p m > p m -i, which certainly holds. □ 

The above lemma suggests an algorithm as follows. Ignoring the constraint in (|23p . and 
with rj > denoting the Lagrangian multiplier associated with the constraint in (|22p . we have 
the following optimality equations: 

F w _ 1 (p e ) = ^(^--^—), £ = l,...,m. (25) 



(Recall from ([To]) , we have Fy/-i(p) = —H'(p).) From (fT4"|) . we know i<V_i(//) is decreasing in 
p; and from Lemma [21 we know the right hand sides of the equations above are also decreasing 
in I. Hence, the solution to these optimality equations, pg, must be increasing in £. 
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To solve these equations, we can start with an rj value sufficiently close to zero, so that 
each of the m equations in (|25j) has a solution. (Note that Fw-i(fj) j when \x — ► oo.) This 
may very well violate the constraint in (|22p . as the resource is priced too low. Hence, we will 
gradually increase rj, to bring down the left hand side of the constraint in (|22|) . until it is 
satisfied as an equality. (Note that since Fw-x{fj) is decreasing in ft, increasing r\ will decrease 
the value of /Vs that solve the equation in (|25p .) As we increase rj, it can happen that for some 
k, even with fi}~ = 0, we still have 

Rk ■= — 7 r < V- (26) 

111 l N 



Then, we set fJLk = 0, as any fik > will only lead to Fw-i(fik) < -^V-i(O). In this case 
we also set fij = for all j < k as well, since Rj < Rk following Lemma [2j Furthermore, 
Hi = ■ ■ ■ = = will remain at zero as rj further increases. (As rj increases, more fj.es may 
become zero, for £ > k.) 

Proposition 3 (i) The optimal switch-over times can be obtained from solving the set of single- 
variable equations in (125j) along with an increasing sequence of r\ (the Lagrangian multiplier) 
until the constraint in (|22p becomes binding. 

(ii) At optimality, for any price class k we have (jp. = if and only if < i] in (|26p ; and if 
fjk = for some k, we must have fjj = for all j < k. 

Proof. What is stated in (i) and (ii) is a summary of the algorithm and the solution it generates. 
Hence, what needs to be argued is the optimality of the solution. To do so, in addition to the 
Lagrangian multiplier rj > 0, let vi > be the Lagrangian multiplier associated with the 
constraint fj£ > fj£—i, for £ = 1, . . . ,m (with := 0). In addition, denote f m +i := 0. Then, 
optimality is reached if and only if (fix, . . . , fj m ) and all the multipliers satisfy the following 
equations: 

iT£F w ^i(fie) = rff- -r ) - i/^ + ^+i, £ = l,...,m; (27) 



^A^ A e+1 

m 

rj 



v e (fji - fie-i) = 0, £=l,...,m; (29) 



and the constraints in (|22p and (|23|) are satisfied. 

As described above, when the algorithm ends, it generates a solution that takes the following 
form: 

:= fJo = fJi = ■ ■ ■ = fik < Hk+i < fJ-k+2 < < fJ-m, (30) 
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for some k E {1, . . . , m}, along with an r\ > 0, while (|22p is satisfied as an equality - and hence 
(|28h holds. In addition, (j23|) follows from f)30|) . Hence, what remains is to verify (I27p and (j29|) . 
These can be satisfied by letting 

v t = 0, ^ = fc+l,...,m; (31) 
Vk = r,(-^--^—)-Tr k F w ^(0); (32) 



i/,- = + r/f- ") -iTjFw-i(0), j = k-l, ...,1. (33) 

First, the z/s above are all non-negative: > follows from f|26|) : and for j < k, Vj > 
follows recursively from (|33h . as well as from (I26h . taking into account Rj < Rk < V for j < k. 
Second, the above z/s, along with the /x's in (|30p . satisfy (|29p . Finally, note that the zero f^'s 
(£ = /€ + 1, . . . , m) in (f3Tj) reduce (f27|) to the equations (with the same t indices) in (p5|) used 
by the algorithm; and substituting ([32]) and (|33l) into ([271) recovers /ij = obtained by the 
algorithm for j < k. □ 

4 The Switch-Over Policy: Random Batch Size 

We now extend the switch-over policy of the last section to allow random batch sizes. In this 
case, while the structural properties in Proposition Q] do not apply, the asymptotic optimality 
results in the next section does provide some justification to the switch-over policy. 

4.1 Homogeneous Batches 

To facilitate the derivation, we start with assuming that all orders independently follow an 
identical batch-size distribution that is independent of the prices, deferring the case of price- 
dependent batch sizes to the next subsection. Specifically, the batch size distribution is denoted 
by qj = P{Q=j}, forj = 1,...,W. 

With the initial inventory of W units, we apply the switch-over policy in [0, t\]. The cumu- 
lative revenue collected by time t\ is: 

Pl E[W -z T M Nl w], 

where, 

z T := (0,0,..., 0,1), w T :=(0,l,2,...,H/); 
Ni := Ni(t%) follows the Poisson distribution with mean Aiii, and M denotes the following 
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probability transition matrix: 



M 



/ 1 
qi 1 - qx 
Q2 qi \-q\- q 2 









qw J 



(34) 



\ Qw Qw-i QW-2 ••• 1 — Ql — 

That is, the number of remaining units in the system, embedded at the arrival epochs of 
orders of price p\, is a Markov chain with the probability transition matrix M. Note that the 
dimension of M is W + 1, corresponding to the dimension of the state space of the Markov 
chain, {0,1, ...,W}. 

Recall that the generating function of a Poisson variable N with mean /i is: 



E[z 



e 



Hence, we have 



P!E[W - z T M Nl w] = Pl [W - e" w z T e wM w]. 



Here, fi\ follows the notation defined in (I18j) : and the matrix exponent is defined as 

A k 



j=0 



for any matrix A. 

Similarly, for the next interval (ii,i2]> when the acceptance includes both price classes pi 
and p2, the expected revenue is (using the notation of the last section): 

pi 2 E[z T M - z T Af 7Vl+iVl2 w] 
= pi 2 [z T {e-^e^ M - e-^e^ M )w]. 

Comparing the above with the objective function in Q, we know the only difference is that 
the terms there, 

E(W - N%) + and E(W - iVi - N 12 ) + 

are now replaced by 

z T e - We AtiAf w and z T e - M 2 e //2M w _ 

With this in mind, we further denote 

G(ji) := z T e-»e» M w, (35) 
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which is analogous to H(/jl) of the last section. The derivatives of G{fj) can be derived as 
follows: 



G'(fi) = -z T e-»(I - M)e" M w, G"(jj) = z T e~^(I - M) V M w. 



(36) 



Note that the spectral radius of the matrix M is unity. Hence, G'(fi) < 0; i.e., G(fj.) is a 
decreasing function. Furthermore, G(fi) is a convex function, since G"(fi) > 0. 

We can formulate the optimization problem as follows, with m, £ = l,...,m, as decision 
variables: 



in y^-KeGine); s.t. V7- )n t <T, < /i X < • • • < yu m . 



mm 



(37) 



Here, same as in (|2Up. 



2/^ : — h — tt-i 



\H — fH-l 



1, ...,m; 



with hq := 0. 

To solve the optimization problem above, we can follow the same approach as in the last 
section; specifically, we can solve the following optimality equations via gradually increasing 
the Lagrangian multiplier r) until the resource constraint becomes binding: 



zV'g-Mje^'w = iL(J_ _ _L), £ = l,...,m. 
Note that both Lemma [2] and Proposition [3] still apply here. 



(38) 



4.2 Price-Dependent Batches 

Now, suppose each price pi, i = 1, . . . , m, is associated with a batch size Qi, with the distribution 
= j) = f° r J = 1) •••> W. The batch sizes are i.i.d. among orders of the same price, and 
independent among orders of different prices. Here, instead of a single probability transition 
matrix M, we have m such matrices, one for each price pi\ 



Mi 



( 1 o 

On 1 — Oil 
@i2 On 1 



V 






Oil — 0i2 
%,W-2 









1 



7,1 



\ 



iW / 



(39) 



Consider the second time interval {t\ , t%] ■ Since orders of both prices pi and p2 are accepted, 
the two Poisson streams will be combined, resulting in the following transition matrix 

T 2 := -HAiMi + A 2 M 2 ). 
A 2 
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Analogously, denote Ti := M\. The expected revenue over this time interval is 

pi2E[z Mj w-z M 1 1 2 wj 



In general, denote 



Pi2[z T (e- w e wri - e-^ 2 e^ iri e^ 2 ^ l)r2 )w]. 



Replace G(/j,e) in the earlier special case of independent batch sizes by the following: 

e 

9e(ye) ■= e"^exp(^A fc y fc r fc ) and G e (y e ) := z T g £ (y e )w. (40) 
fc=l 

(Notice that from ([18]) , we have m — = Agyi-) 

The optimization problem can now be expressed as follows, with yg, £ = 1, m, as decision 
variables: 

m m 

min ^J-7r^G^(y^); s.t. } \ yt < T; yi > 0, £=l,...,m. (41) 

v i=i e=i 

From ()40p . we know that ge(ye) is convex in y; hence, the problem in (|4ip is a convex program 
(albeit no longer separable) and as such can be solved by standard algorithms. 

4.3 A Numerical Example 
Example 1 Consider four price classes: 

Pl = l, P2 = 0.8, p 3 = 0.65, p 4 = 0.45, 
with Poisson arrivals at the following rates: 

\ 1 = 0.2, A 2 = 0.3, A 3 = 0.1, A 4 = 0.4. 

The batch size Q is homogeneous among the four classes, following a discretized exponential 
distribution with mean 12. (Specifically, P(Q = n) = e A ( ra+1 ) — e An , for n = 0,1,2..., and 
1/A = 12.) Let T = 20, and let W vary from 1 to 100. 

We compare the performance of the switch-over policy against the optimal policy (from 
DP) and a first-come-first-served (FCFS) policy (i.e. supply any order on arrival as long as 
there are units available). The objective values corresponding to the three policies are plotted 
in Figure 1. The relative errors of the switch-over policy and the FCFS policy with respect 
to the optimal policies are plotted in Figure 2. The performance of the switch-over policy is 
remarkably close to the optimal policy, while the performance of the FCFS policy, as expected, 
deteriorates quickly as W increases. □ 
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5 Asymptotic Optimality of the Switch-Over Policy 



In this section, we establish the fact that the switch-over policy is asymptotically optimal, in the 
sense that as W increases to infinity, the relative error between the switch-over policy and the 
optimal policy (via dynamic programming) will go to zero. In fact, we show the absolute error 
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is no more than 0(VW); hence, the rate of convergence is 0(y/W). This type of asymptotic 
optimality is in the same spirit as other recent studies, e.g., [14] . Similar results have been 
established for various revenue management problems, see, e.g. [31 [19] . The result here 
applies to both models in the last two sections; for ease of exposition, we shall focus on the 
model in $3l 

Intuitively, when the available inventory W — ► oo, we should expect the switch-over policy 
to be optimal in the following two cases: (a) if the decision horizon T remains a constant or is 
of lower-order than W, then supplying all orders (i.e., first-come-first-served) would be optimal; 
and (b) if T is of higher order than W, then supplying the highest bid only (orders priced at 
pi) should be optimal. In both cases the policies can be represented as special cases of the 
switch-over policy. Below, we will formally confirm the intuition behind these two special cases, 
and also prove the result in the non-trivial case in which T — > oo in the same order as W. 

We start from an upper-bound on the performance of the optimal policy. 

Lemma 4 The objective value under the optimal policy (as derived from the dynamic pro- 
gramming), denoted V*, satisfies the following inequality: 

V* < (X lPl + ■■■ + X kPk )T + X k+lPk+1 t, (42) 

where k = 1, m — 1 and t £ [0, T) are such that A k T < W, A k+ {T > W, and 

A k T + X k+1 t = W. (43) 

If A]T > W, then V* < p x W. If A m T < W, then V* < pi m A m T. 

Proof. Note that V* is maximal among all non-anticipative policies; in particular, at each time 
t all future order arrivals and their prices are only known in distribution. Now, suppose at time 
t = we know all arrivals over (0, T] and their prices deterministically. Specifically, suppose 
over (0.T], there are N\ orders of price p\, N2 orders of price P2, and N m orders of price 
p m . Then, we will obviously use the W available units to first supply the N± orders of price p\\ 
and then, if there is anything left, supply the N2 orders of price p2\ and so forth. Clearly, no 
non-anticipative policy can do better than this one, which yields an objective value of 

m—l 

Pl [W A N 1 ]+p 2 [(W - Ni)+ A N 2 ] + ■ ■ ■ + Pm [{W - Y, N k) + A N m ] 

k=l 

m—l m 

= PiW - (pi - p 2 )(W - N x )+ (p m _x - p m )(W - N k) + - Pm(W - N k) + - 

k=l k=l 
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Taking expectation on the right hand side above and passing the expectation into (-) + further 
increases its value (since the function is convex). Hence, we have 



m—l 

V* < p l W-{ Pl - P2 ){W-X l T)+ (Pm-l - Pm)(W - W 

k=l 

m—l 

-p m (W- A fe T)+ 

k=l 

= PiW - (pi - P2)(W - \\T) (p fc - Pk+1 )(W - AiT A fc T) 

= pk+iW + (p a - p fc+ i)AiT + (p2 - pfc+i)A 2 T H h (pjfe - Pfc+i)A fc T 

= (Aipi H h \kPk)T + Afe+ipfc+it, 

where the last two equalities follow from the relation in (|43|) . 

Following the same reasoning also leads to the bounds in the two cases when A±T > W or 
A m T <W. □ 

Next, we derive a lower bound on the switch-over policy. 

Lemma 5 Let V sw denote the objective value under the best switch-over policy. Then, letting 
k and t be defined as in Lemma [H we have 

^ sw > PikW - (pik - pi, k+ i)E[W -N(A k (T-t))} + 

-p ljk+1 E[W-N(A k T + X k+1 t)]+; (44) 

and 

V sw > Pl E[N{W) A W] if AiT > W, 
V sw > Plm E[N(A m T)AW] if A m T < W. 

Proof. Clearly, a feasible switch-over policy, realizable by setting t± = ■ ■ ■ = t k = and 
tk+l = T — t, is to accept the top k price classes throughout the horizon [0, T] and the class 
k + 1 over [T — t, T], the last t time units. Hence, 

V sw > Plk E[N(A k (T-t))AW]+p hk+1 E{N(A k+1 t)A[W-N(A k (T-t))} + } 
= PikW - Plk E[W -N(A k (T-t))} + 

+ Vlyk+l E[W - N(A k {T - t))]+ - Pl , k+ iE[W - N(A k (T - t)) - N(A k+1 t)} + 
= PikW - ( Plk - p 1M1 )E[W - N(A k (T - t))]+ - Pl>k+1 E[W - N(A k T + X k+1 t)] + 
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(Recall, pik is the average price among classes 1 through k, defined in flSJ), and pi^+i is similarly 
defined.) If A±T > W, then serving class 1 only throughout [0, T] results in 

V sw > piE[JV(AiT) AW}> Pl E[N(W) A W]. 

If A m T < W, then serving all m classes throughout [0,T] leads to V sw > p lm E[N(A m T) A W\. 
□ 

We shall also need the following result: For the Poisson variate with mean a, denoted N(a), 

E[a - N(a)] + ~ J±, a -> oo; (45) 

which can be directly verified via the Poisson distribution (along with the Stirling formula), or 
via the normal approximation of the Poisson distribution. 

Proposition 6 The switch-over policy is asymptotically optimal, in the sense that V sw /V* — > 
1 when W — * oo. 

Proof. To start with, suppose while W — * oo, T remains a constant. This corresponds to the 
case of A m T < W in Lemmas H] and [5j and we have 

PimAmT > V* > V sw > pi m E[N(A m T) A W] 

-> p lm E[N(A m T) =p lm A m T, 

where the limit follows from monotone convergence as W — ► oo. Hence, in this case we have 
V* = V sw = Plm A m T. 

Now, suppose T — ► oo; and without loss of generality, suppose as a function of W, T(W) 
satisfies the following: 

T(W) 

< c := liminf < oo. 

W 

That is, we allow T to be of lower or higher order than W, as well as of equal order to W. 
First, suppose 

Ar 1 < c < A m x . Then, we can assume the relation between W and T in (|43|) to 
hold, for some k. Note that the upper bound in (|42p can be written as: 

(Aipi H h A fc p fc )T + A fc+ ip fc+ it 

= A fc pi fc r + Afc+iPfc+it 

= yifc^F - (pifc - pfc + i)Afc + ii. 

Hence, combining (|42p and (jUJ), we have 
< y* - y sw 

< -(pi* - Pfc+i)A fc+ it + (pi fc - pi )fc+ i)E[W - iV(A fc (T - *))]+ 

+Pi, fc+ iE[PF-Af(A fc T + A fc+1 t)] + . (46) 
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Since 

Pifc(Afc+i - Afc + i) = Aipi + • • • + XkPk = Pi,fc+iAfc+i — pk+iXk+i, 

we can write 

(Pik -pk+i)X k +it = (pik -pi,k+l)[W - A k (T - t)\. 

Furthermore, 

E[W-N(A k (T-t))] + = E[W-A k (T-t) + A k (T-t)-N(A k (T-t))] + 

< W-A k (T-t) + E[A k (T - t) - N(A k (T - *))]+, 

where the inequality follows from W > A k (T — t). Hence, the first two terms on the right hand 
side of (pR)]) can be combined to yield the following: 

< V* - V sw 

< (Pik ~ Pi, k +i)E[Ak(T -t)- N(A k (T - t))]+) + Pl , k+ iE[W - N(W)} + . 

From (|45p . we know the two terms on the right side above are of order 0(VT) and 0(VW), 
respectively. On the other hand, V* is clearly of order OiW) — both the upper and lower 
bounds in Lemmas U] and [5] are of order 0(W). Hence, dividing both sides above by V* and 
letting W -> oo results in V sw /V* -> 1. 

Next, consider the case of c > A^ 1 , which corresponds to the case of A\T > W . From 
Lemmas H] and O we have 

0<V*-V sw < Pl W- Pl E[W AN(W)} + 

= Pl E[W - N{W)} + ~ 0(VW). 

In this case, V* is still of order 0(W); and hence, y sw /V* —>■ 1. Finally, in the case of c < A" 1 , 
which corresponds to A m T < W, we have 

0< V *-V SW < p lm A m T - Plm E[W AiV(A m T)]+ 

< p\ m A m T — pi m E[A m T A N(A m T)] + 
= pi m E[A m T - N(A m T)}+ ~ 0(VT). 

In this case, V* is of order O(T); hence, y sw /V* — > 1 follows when T — ► oo. □ 

Example 2 We continue with the example in ^4.3^ but suppose the batch size Q follows neg- 
ative binomial distributions: 

P[Q = k]= (* + r _~% r (l-p) k , fc = 0,l,2,-.. ; 
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w 


(r,p) 


Optimal 


Switch % off 


Equal % off 


20 


(4,0.33) 
(8,0.5) 


17.59 
17.68 


17.47 0.67% 
17.50 0.99% 


17.34 1.37% 
17.28 2.27% 


40 


(4,0.33) 
(8,0.5) 


34.39 
34.60 


34.16 0.67% 
34.26 0.97% 


33.99 1.18% 
34.14 1.34% 


60 


(4,0.33) 
(8,0.5) 


50.08 
50.39 


49.62 0.92% 
49.84 1.09%) 


48.74 2.69% 

A f\ 1 A O AO 0"/ 

49.14 2.48% 


160 


(4,0.33) 
(8,0.5) 


101.43 
102.38 


100.87 0.55% 
101.84 0.52% 


72.85 28.18% 
72.72 28.97% 


180 


(4,0.33) 
(8,0.5) 


104.54 
105.48 


104.37 0.16% 
105.31 0.16% 


72.69 30.47% 
73.14 30.66% 


200 


(4,0.33) 
(8,0.5) 


105.74 
105.78 


105.69 0.05% 
105.76 0.01% 


72.73 31.20% 
72.56 30.82% 



Table 1: Performance of the policies as W increases (with fixed T). 



W,T 


(r,p) 


Optimal 


Switch 


% off 


W,T 


(r,p) 


Optimal 


Switch 


% off 


20, 20 


(4,0.33) 
(8,0.5) 


17.59 
17.68 


17.47 
17.50 


0.67% 
0.99% 


40, 40 


(4,0.33) 
(8,0.5) 


36.74 
36.87 


36.55 
36.60 


0.52% 
0.73% 


60, 60 


(4,0.33) 
(8,0.5) 


55.62 
55.71 


55.38 
55.41 


0.43% 
0.54% 


80, 80 


(4,0.33) 
(8,0.5) 


75.11 
75.14 


74.95 
74.97 


0.21% 
0.23% 



Table 2: Performance of the switch-over policy with simultaneous increase of both W and T. 



and hence, 

m _ r£zrt , Var[Q] = 

P P Z 

We choose (r,p) to be (4,0.33) and (8,0.5). While both distributions have the same mean 8, 
their coefficients of variation are 3 and 2, respectively. 

In Table [H we compare the performance between the optimal policy and the switch-over 
policy, with the switch-over times optimized (the column under "Switch"). Fixing T = 20, we 
increase the value of W. The relative error between the two policies first has a slight increase, 
and then quickly decreases as W becomes large (8-10 times the value of T). In addition, we list 
in the table the performance of a switch-over policy with equally spaced switch-over times (the 
column under "Equal"), i.e., = T/4 = 5 for all i. The results indicate that the performance 
of this policy deteriorates rather severely as W increases. That is, it is crucial to optimize the 
switch-over times. 

Next, in Tabled we increase T simultaneously with W. The relative error of the switch-over 
policy appears to decrease monotonically. □ 
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6 Pricing Models 



As mentioned in the introductory section, an important motivation for us to study the switch- 
over policy is to use it as a means to solve optimal pricing problems. (In this sense, the prices 
derived below are only optimal with respect to the switch-over policy.) Suppose, instead of 
assuming all prices are given, and ordered as in ([7]), it is now our decision to come up with the 
m — 1 "discount" prices, p 2 > ■ ■ ■ > p m - The original price p\ is still given, and p\ > p 2 . (The 
case when p\ is also a decision variable is discussed at the end of this section.) 

Here, m is also assumed to be given. That is, we limit the number of price takedowns that 
can take place over the horizon. Furthermore, we assume the time horizon is divided into m 
equal segments, with one price for each segment. Without loss of generality, assume T = m, 
hence each segment is of unit length. Note that the equal-length segments do not contradict the 
earlier models, where the time lengths yi corresponding to accommodating different prices are 
decision variables, and hence may vary in t. It can happen that in the derived pricing solution 
we have pi = pi+±, for instance, then the price pi will apply to two time segments instead of 
one. 

Consider the model in §4.11 i.e., batch Poisson order arrivals, with the batch size independent 
of the price (hence, we are not modeling phenomena such as quantity discount); however, let the 
arrival rate be a decreasing function of the price, \(p), which is also assumed to be differentiable. 

Following the analysis that leads to the problem formulation in (j37H . and noticing that 
pu becomes pi here, since now there is only a single price in each time segment, we have the 
following optimization problem: 

max px[W - G(m)] + p 2 [G(m) - G(p, 2 )] H h p m [G(/i m _i) - G(fj, m )] 

= PlWi - (pi - p 2 )G(fli) (Pm-l - Pm)G{n m -l) ~ PmG(Hm), (47) 

where (p 2 , ...,p m ) are decision variables, satisfying the ordering in ([7]); and 

Hi = Aj = A(pi) H hA(pi), i = l,...,m. 

Equivalently, we can solve the following minimization problem: 

min (pi - p 2 )G(m) H h (p m -l - Pm)G{ii m -x) + p m G{n m ), (48) 

P2,...,Pm 

S.t. pi > p 2 > ■ ■ ■ > Pm > 0. 

The above problem, although no longer separable, can still be solved by a standard nonlinear 
(convex) programming algorithm. Below, we present an approximation algorithm that is easy 
to run and appears to work quite well. 
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Apply a transformation of variable: let the new decision variables be 

r m --=Vm\ n := pi - p i+ i, i = l,...,m-l. (49) 

The optimization problem in (|48|) then becomes as follows: 

m 

min y^riG(m), s.t. rH hr m =pi; r» > 0, z = l,...,m. (50) 

i=i 

Write the derivatives 

Aj A^=A' 1 + --- + A^. 

We can derive, for any i and j, 

g/fi _ A/ 

Or, iAj " 

This follows from noticing that when j > j , pi = T{ + • • • + r m does not involve rj. 
Therefore, the optimality equations are: 

m 

G (H) + Yl nG'(fn)A' iAj =77, j = 1, m. (51) 

i=l 

Taking the difference between two consecutive equations above, we have 

m 

G(nj) - + A;- ^ nG'^i) = 0, j = 2, m; 

From the above, we can derive: 

Now, ignoring the second term on the right hand side above, we have the following approxima- 
tion: 

Observe that 

■\? = K r j H h r m ) = A(pi - n r 3 -_i) 

only involves (rx, ...,rj_i); and hence, so does fij. That is, the right hand side of ([53]) . for each 
j, only involves (ri, ...,r-,_i). Therefore, following the recursion, we can relate ..-,r m all to 
ri, and then derive ri through a simple line search via the equation 

n+r 2 -\ \-r m =pi. (54) 
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(W;a,b) 


i = 


12345678 


obj. val. 


(40; 15, 14) 
(40; 40, 37.33) 


Pi 
Pi 


1 0.69 0.69 0.69 0.69 0.69 0.68 0.62 
1 0.63 0.61 


25.64 
22.94 


(40; 15, 2) 
(40; 40, 2) 


Pi 
Pi 


1 0.60 0.60 0.60 0.60 0.60 0.59 0.56 
1 0.55 0.54 


20.77 
19.95 


(40; 2, 1.5) 
(40; 5.33, 1.5) 


Pi 
Pi 


1 0.57 0.57 0.57 0.57 0.57 0.54 0.37 
1 0.52 0.44 


20.46 
20.10 



Table 3: Optimal pricing under linear, exponential and power demand functions. 



In executing this recursion, we have noticed that when p\ is not large enough, n might 
become negative while we solve the equation in (j54|) . This is because the other rj's, j ^ 1, are 
all positive, as evident from (|53p . One way to avoid this from happening is to first replace p\ 
by Cpi, where C is a large positive number. Once all the r/s are derived and r\ is positive, 
divide each of these by C. 

In the following examples, we consider three commonly used functions, the linear, exponen- 
tial and power functions, that model the relation between the arrival rate and the price: 

X(p) = a-bp, X(p) = ae- b f, A(p) = ^; (55) 

where a and b are positive parameters in all three cases. The corresponding derivatives are: 

A'(p) = -b, A'(p) = -b\(p), A'(p) = --A(p). 

V 

Example 3 In this example we choose p\ = 1, and consider the linear, exponential and power 
functions in (|55p with different parameters (o, b), along with different W values. The choice of 
the parameters (a, b) is such that all three functions coincide at p = 1; the linear and exponential 
functions also coincide at p = 0; while the power function coincides with the linear function at 
a point close to the origin. 

We list both the optimal solutions and the approximations following (|53p . In each case, we 
consider both m = 8 and m = 3. The results are summarized in Table El where listed under 
"obj. val." are the original, maximal objective values following (|47p (in particular, including 
the p\W term). 

Example 4 Next, we examine the optimal price reduction with respect to the available inven- 
tory W. We take the exponential demand case above, fix m = 8 and (a, b) = (15,2), while 
changing W. The results are displayed in Table HI 
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w 


i = 


12345678 


obj. val. 


50 


Pi 


1 0.52 0.52 0.52 0.52 0.52 0.52 0.52 


21.28 


30 


Pi 


1 0.73 0.73 0.73 0.73 0.73 0.71 0.61 


19.35 


25 


Pi 


1 0.82 0.82 0.82 0.82 0.81 0.78 0.65 


18.12 


20 


Pi 


1 0.93 0.93 0.93 0.93 0.92 0.86 0.69 


16.43 


15 


Pi 


1111111 0.78 


14.00 


10 


Pi 


1111111 0.88 


9.90 



Table 4: Optimal pricing under various inventory levels. 



(W;a,b) 


i = 


12 3 


obj. val. 


% off opt 


(40; 40, 37.33) 


Pi 
Pi 


1 0.66 0.45 
1 1 1 


21.82 (approx) 
8.00 (non-opt) 


5% 
65% 


(40; 40, 2) 


Pi 
Pi 


1 0.68 0.35 
1 0.90 0.90 


19.10 (approx) 
17.32 (non-opt) 


4% 
13% 


(40; 5.33, 1.5) 


Pi 
Pi 


1 0.68 0.36 
1 1 0.50 


19.84 (approx) 
18.21 (non-opt) 


1% 
10% 



Table 5: Approximations and non-optimal pricing schemes. 



Example 5 Finally, we examine the approximation scheme in (|53j) , which solves the optimality 
equations approximately. We take the above cases under m = 3. We also examine non-optimal 
alternatives that offer different levels of discount at different periods or no discount at all. (For 
instance, in the second case, (1,0.90,0.90) indicates full price in period 1, and 10% discount in 
the remaining two periods.) The results are summarized in Table [H where the last column is 
the percentage off the objective value under optimal pricing. 



From the above results, we observe that 

(a) reducing the number of price markdowns from 7 (m = 8) to 2 {m = 3) has a rather minor 
effect on the objective values; 

(b) with more inventory available for sale, price reduction becomes more substantial and 
starts earlier, as expected; 

(c) the approximation scheme in (|53p performs quite well in all three cases; 

(d) applying the optimal pricing results in a substantial advantage over other ad- hoc schemes. 

The model and analysis discussed above extend readily to the case where, in addition to 
the discount prices, the original price p\ is also a decision variable. The optimization problem 
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now becomes 

m 

max p\W — y riG(ni) (56) 

pi,r 

i=l 

s.t. n H hr m >pi; pi > 0; r» > 0, i = 1, ...,m. 

The Lagrangian is 

m 

Pl w - nG{m) + r]{n + ■■■ + 

r m PX)i 

i=l 

with 7] being the multiplier. Therefore, the optimality equations in (|52p still apply; and in 
addition, we have rj = W (from setting the partial derivative with respect to p\ to zero). 
Similarly, the recursion in (|53|) also applies, assuming p\, as well as r±, is given; and n is still 
obtained from the summation constraint: r± + • • • + r m = p\. (That this constraint must be 
binding follows from the fact that the term, YliLi r iG{f^i), is increasing in (7*1, r m ), which 
can be directly verified.) Finally, p\ can be obtained from another line search to enforce r] = W . 

7 Concluding Remarks 

The models discussed here can be extended in a number of ways. First, the demand function 
in the pricing model as represented by the Poisson arrival rate can depend on time (period), i, 
as well as on price pi, to take the form \(i, P i). For instance, a discount that takes place earlier 
might attract more (or, less) demand. The results in £}6] will continue to hold, since the only 
change needed is from m = X(pi) to /i, = \(i, P i). 

Second, the available inventory W can be made a decision variable too. This amounts to 
constructing a newsvendor problem on top of the switch-over or pricing models developed here. 
It is readily verified that all the objective functions (for maximization) involved are concave 
in W. Hence, with the addition of replenishment and salvage costs, the resulting newsvendor 
model can be routinely solved, once the optimal switching points or optimal prices are obtained 
(for each given W). 

Third, in the switch-over policy, we can incorporate a service measure such as the acceptance 
rate for each price class k: recall, it will not be accepted until the time interval i,tjfc]. 
Consider the model in §4.11 for instance. We know the total expected number of accepted 
units, over the time interval (tfc-i,i&] is G(^fc_i) — G(fik), of which the share of class k is 
proportion to Afc/A^. Hence, the expected number of accepted units for class k over the entire 
horizon (0, T] is: 

m . 

i=k l 
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The above divided by the expected number of class k arrivals over the horizon, AfcTE(Q), is 
what we call average acceptance rate for class k, denoted We can hence derive after the 
optimal switch-over points are derived; or, include a minimal requirement for as a constraint. 
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